1. Field of the Invention
This invention relates to an image shake detecting device which is highly suited for the automatic image stabilizing device of a video camera or the like and also to a device for controlling the image stabilizing device.
2. Description of the Related Art
In the image sensing optical apparatuses of varied kinds including video cameras, etc., irrespective as to whether they are adapted for industrial instrumentation, or consumer appliances, shaking of an image not only hinders easy image sighting but also degrades image recognizing accuracy. In the case of the video camera, for example, the camera is often operated while the operator is walking or while the camera is on a moving vehicle. In such a case, it is inevitable to have a sensed image shaken by a shake of the camera according to the photographing conditions or the object to be photographed.
To solve this problem, there have been proposed image shake detecting devices adopting varied methods. In one of such methods, the movement of the camera is physically detected by means of an acceleration sensor (an angular velocity sensor) and an optical system is compensated for the movement according to the direction and the degree of of the movement. In another conceivable method, the parallel moving extent of of the whole image plane is detected through a video signal and is expressed in a movement vector. Then, the optical system is compensated on the basis of the vector.
In accordance with the method of using the acceleration sensor, the size of the device becomes larger thus requiring increases in space and weight. Besides, it results in a complex structural arrangement. This method is therefore hardly suited for a home video camera such as a camera-incorporating type video tape recorder which must be compact in size and light in weight.
As regards the method of computing and obtaining the movement vector of the image plane from the video signal, some camera movement that is intentionally caused by the operator might be mistakenly detected for a shake of the image. The device also would respond to a movement of the object which is in realty not a shake of the image. That method thus also has a serious problem.
The above-stated known image shake detecting devices include, for example, an image stabilizing camera which is disclosed in Japanese Laid-Open Patent Application No. SHO 61-248681. The camera of this kind is arranged as follows: an optical image is converted into an electrical signal by means of an image sensing system which consists of a lens system and a photo-electric converting element. A TV image signal is obtained through a signal processing operation performed in a given manner on the electrical signal by a signal processing circuit. The image signal thus obtained is supplied to an image shake detecting circuit as well as to a monitor. A correlation between two image planes obtained at a given time interval is detected from the image signal to find the degree and the direction of any image shake. Then, a driving control circuit and a motor are operated to control and move the lens system to offset the image shaking on the basis of the result of detection. The camera is thus arranged to be capable of obtaining a stable image even when the camera shakes.
However, the image shake detecting device which is arranged in this manner is incapable of making a discrimination between the movement of an object occurring only in a part of the image plane and a shake of the whole image plane. To solve this problem, the image shake detecting sensitivity of the device must be arranged to vary for the different areas of the image plane.
In connection with this problem, an image shake detecting device has been proposed as disclosed in an article entitled "About an Image Plane Shake Compensating Device," appeared in "The Technical Report" of the Television Society, Vol. 11, No. 3, p 43 to 48, PPOE, '87-12 (May, 1987). In the case of that device, the whole image plane is divided into 140 blocks of areas. A shake detection switch is arbitrarily turned on or off for each of these areas and an image shake is detected only from the areas for which the shake detection switch is turned on in accordance with a representing point matching method.
In accordance with the arrangement of this image shake detecting device, however, an image to be used as reference must be temporarily stored at a frame memory with its varied density values kept intact. To meet this requirement, the device necessitates the use of an analog-to-digital (hereinafter referred to as A/D) converter and a memory of a relatively large capacity. In addition to this shortcoming, the device is arranged to have one image plane superposed on another by staggering them to a certain degree of vector and to find out a vector that gives the highest degree of coincidence. Therefore, the operation of the device includes a large amount of computation. The device thus necessitates a circuit arrangement on a large scale and requires a long computing time.
Besides, it has been extremely difficult to have the image shake detecting device incorporated in a compact video camera which must be capable of carrying out real-time processing and must be arranged in a compact circuit arrangement, because: The above-stated device of the prior art necessitates the use of a large circuit arrangement including the A/D converter, the frame memory, a computing circuit, etc. and a long processing time.
Further, a control system which is provided for actually effecting shake compensation by driving a photo-taking optical system on the basis of information on a shake detected by the above-stated image shake detecting devices has been arranged as follows: Various methods have been known for correcting a displacement of an image. In one of these varied methods which has recently been developed, there are provided optical compensation members such as a lens which is arranged to have its optical axis tiltable by a motor and a variable apex-angle prism. With an image obtained through an optical system which includes these optical compensation members, the deflected degree and the size of the image to be compensated on the basis of the image displacement are detected from the image. Then, the so-called feedback control is performed to deflect the image by driving the above-stated optical compensation members in accordance with information on the results of detection.
In the case of the device of this kind, a variable apex-angle prism which is arranged to have its apex angle variable is employed as an optical image deflecting means. A motor is used for driving the prism. The deflected degree of the image is detected in the following manner: The images of at least two TV camera image planes having a time difference between them are compared with each other. The deflection of the variable apex-angle prism is then controlled in such a way as to lessen displacement occurring between the two images compared, so that the images can be obtained always in a coinciding state. Further, in cases where the object to be photographed is traced (tracked) or where the object moves, the variable apex-angle prism is driven in such a way as to lessen the degree of the positional change of the object's image within the image plane. In a case where the image is shaken by a shake of the camera caused by something, the image is stabilized by driving the variable apex-angle prism in the same way as in the case of tracing the object to lessen the degree of the positional change of the object's image. Therefore, in these cases, the control is thus performed virtually in the same manner.
However, an interlaced scanning method is used for the TV camera or the like in general. In the case of the TV camera of the NTSC system, one image plane is formed and transmitted for every period of 1/60 sec (one field period). In a case where the feedback control method is employed for detecting the state of the image obtained by such a TV camera, the data of the feedback system consists of samples dispersed at intervals of at least 1/60 sec. Besides, the feedback action delay involves a period of time required for a detection process plus a period of time required for image plane transmission (about 1/60 sec). The control output of the device, therefore, is unstable for any quick movement of the object. The response characteristic of the device is poor. In a case where the motor used for driving the optical compensation member has a good rise characteristic, a feedback coefficient exceeding 1 tends to cause oscillation. A further drawback of the prior art device resides in a poor frequency characteristic. These problems stem from the fact that the control is performed by using the control algorithm of a continuous time system while sampling by detecting means has a dispersive time delay.
Further, there has been proposed an apparatus which uses an image shake detecting device for objects appearing within an image plane and is arranged to trace (track) a moving object and to continuously perform control for accurate automatic focusing and accurate automatic exposure control. For example, apparatuses of this kind have been disclosed in U.S. patent applications Ser. No. 737,163 filed on May 23, 1985; Ser. No. 106,427 filed on Oct. 8, 1987; Ser. No. 154,078 filed on Feb. 9, 1988; Ser. No. 237,511 filed on Aug. 26, 1988; Ser. No. 240,915 filed on Sep. 6, 1988; Ser. No. 258,692 filed on Oct. 17, 1988; and Ser. No. 264,204 filed on Oct. 28, 1988. However, as mentioned in the foregoing, it is difficult to accurately make a discrimination between the movement of only one object in a part of the image plane and the movement of the whole image plane due to the movement of the camera. In accordance with the above-stated method disclosed in Japanese Laid-Open Patent Application No. SHO 61-248681, the degree of accuracy would lower if there is no luminance difference because the method utilizes a difference in luminance between the background and the object to be photographed. Further, in accordance with the technique disclosed in the above-stated Technical Report of the Television Society, it becomes difficult to accurately determine the areas in a case where a plurality of objects come to move. In other words, each of the methods of the prior art has a disadvantage as well as its advantage. Therefore, with the prior art image shake detecting devices incorporated in a compact video camera in accordance with these methods, they are incapable of adequately coping with every image condition.
With respect to the image shake detecting device of this kind, further examples have been disclosed in U.S. patent application Ser. No. 855,732 filed on Apr. 25, 1986, U.S. patent application Ser. No. 880,152 filed on Jun. 30, 1986, etc. These patent applications disclose an image stabilizing camera which is arranged to compensate the optical axis of the lens system by detecting an image shake through the edge component of the image of an object to be photographed; and an arrangement to recognize an image through a histogram which represents the distribution of the feature of the image.